Understanding the landscape of web-based medical misinformation about vaccination.

Given the high rates of vaccine hesitancy, web-based medical misinformation about vaccination is a serious issue. We sought to understand the nature of Google searches leading to medical misinformation about vaccination, and guided by fuzzy-trace theory, the characteristics of misinformation pages related to comprehension, inference-making, and medical decision-making. We collected data from web pages presenting vaccination information. We assessed whether web pages presented medical misinformation, had an overarching gist, used narrative, and employed emotional appeals. We used Search Engine Optimization tools to determine the number of backlinks from other web pages, monthly Google traffic, and Google Keywords. We used Coh-Metrix to measure readability and Gist Inference Scores (GIS). For medical misinformation web pages, Google traffic and backlinks were heavily skewed with means of 138.8 visitors/month and 805 backlinks per page. Medical misinformation pages were significantly more likely than other vaccine pages to have backlinks from other pages, and significantly less likely to receive at least one visitor from Google searches per month. The top Google searches leading to medical misinformation were "the truth about vaccinations," "dangers of vaccination," and "pro con vaccines." Most frequently, pages challenged vaccine safety, with 32.7% having an overarching gist, 7.7% presenting narratives, and 17.3% making emotional appeals. Emotional appeals were significantly more common with medical misinformation than other high-traffic vaccination pages. Misinformation pages had a mean readability grade level of 11.5, and a mean GIS of - 0.234. Low GIS scores are a likely barrier to understanding gist, and are the "Achilles' heel" of misinformation pages.

Gist Inference Scores Predict Cloze Comprehension "In Your Own Words" for Native, Not ESL Readers.

Three patient education texts from the National Cancer Institute (NCI) were subjected to a Coh-Metrix analysis, then further analyzed to obtain Gist Inference Scores (GIS), a new measure of the likelihood that readers will make appropriate inferences about a text's bottom-line meaning. In the GIS formula, the Coh-Metrix psycholinguistic variables referential cohesion, deep cohesion, and latent semantic analysis (LSA) verb overlap increase GIS because cohesive texts that describe related actions are likely to induce gist representations. The Coh-Metrix variables word concreteness, imagability for content words, and hypernymy for nouns and verbs are negatively weighted because they tend to promote verbatim mental representations. NCI texts were modified for a cloze procedure with every tenth word replaced by a blank starting with the second sentence. Participants in two studies received all three cloze-modified texts. Fuzzy-Trace Theory suggests that people are more likely to comprehend high GIS texts "in their own words," and thus fill-the-blanks with multiple words that differ from those omitted by the cloze procedure expressing comparable meaning. In Study One, non-native English speakers appropriately filled blanks with different words at the same rate for all three texts of low-, medium-, and high-GIS. In Study Two, replicating previous findings, for high GIS texts, native English speakers filled blanks appropriately with words other than those removed significantly more often than for medium- or low-GIS texts. High GIS texts apparently afford readers more semantic and lexical flexibility, but non-native English speakers may be ill-equipped to capitalize on this characteristic of high GIS texts.

Gist Inference Scores predict gist memory for authentic patient education cancer texts.

OBJECTIVES: Develop a tool to evaluate and improve written medical communication to patients. Determine how effectively Gist Inference Scores (GIS) predict comprehension of patient education texts independently of health literacy. Explicate the text characteristics and psychological mechanism underlying GIS. METHODS: For study 1, a nationally representative sample of older women (N = 61) completed a fill-in-the-blank comprehension task on authentic National Cancer Institute (NCI) texts of varying GIS levels. In study 2, participants (N = 198) read NCI texts (high or low GIS) then recalled what they read. RESULTS: Study 1 showed that a higher percentage of different words yielding semantically similar sentence meaning were used to correctly fill-the-blanks on high GIS texts and there was no significant interaction with health literacy. In study 2, a greater proportion of decision-relevant information was recalled for high GIS texts. CONCLUSIONS: GIS predicts the likelihood that readers will form gist representations of medical texts on free recall and fill-in-the-blank tasks. High GIS texts allow for more semantic flexibility to mentally represent the same meaning, and more strongly emphasizes gist rather than verbatim representations. PRACTICAL IMPLICATIONS: GIS provides medical communicators with an automated and user-friendly method to evaluate medical texts for their ability to convey the bottom-line meaning.

Automatic Evaluation of Cancer Treatment Texts for Gist Inferences and Comprehension.

Background. It is difficult to write about cancer for laypeople such that everyone understands. One common approach to readability is the Flesch-Kincaid Grade Level (FKGL). However, FKGL has been shown to be less effective than emerging discourse technologies in predicting readability. Objective. Guided by fuzzy-trace theory, we used the discourse technology Coh-Metrix to create a Gist Inference Score (GIS) and applied it to texts from the National Cancer Institute website written for patients and health care providers. We tested the prediction that patient cancer texts with higher GIS scores are likely to be better understood than others. Design. In study 1, all 244 cancer texts were systematically subjected to an automated Coh-Metrix analysis. In study 2, 9 of those patient texts (3 each at high, medium, and low GIS) were systematically converted to fill-the-blanks (Cloze) tests in which readers had to supply the missing words. Participants (162) received 3 texts, 1 at each GIS level. Measures. GIS was measured as the mean of 7 Coh-Metrix variables, and comprehension was measured through a Cloze procedure. Results. Although texts for patients scored lower on FKGL than those for providers, they also scored lower on GIS, suggesting difficulties for readers. In study 2, participants scored higher on the Cloze task for high GIS texts than for low- or medium-GIS texts. High-GIS texts seemed to better lend themselves to correct responses using different words. Limitations. GIS is limited to text and cannot assess inferences made from images. The systematic Cloze procedure worked well in aggregate but does not make fine-grained distinctions. Conclusions. GIS appears to be a useful, theoretically motivated supplement to FKGL for use in research and clinical practice.

A theoretically motivated method for automatically evaluating texts for gist inferences.

We developed a method to automatically assess texts for features that help readers produce gist inferences. Following fuzzy-trace theory, we used a procedure in which participants recalled events under gist or verbatim instructions. Applying Coh-Metrix, we analyzed written responses in order to create gist inference scores (GISs), or seven variables converted to Z scores and averaged, which assess the potential for readers to form gist inferences from observable text characteristics. Coh-Metrix measures reflect referential cohesion and deep cohesion, which increase GIS because they facilitate coherent mental representations. Conversely, word concreteness, hypernymy for nouns and verbs (specificity), and imageability decrease GIS, because they promote verbatim representations. Also, the difference between abstract verb overlap among sentences (using latent semantic analysis) and more concrete verb overlap (using WordNet) should enhance coherent gist inferences, rather than verbatim memory for specific verbs. In the first study, gist condition responses scored nearly two standard deviations higher on GIS than did the verbatim condition responses. Predictions based on GIS were confirmed in two text analysis studies of 50 scientific journal article texts and 50 news articles and editorials. Texts from the Discussion sections of psychology journal articles scored significantly higher on GIS than did texts from the Method sections of the same journal articles. News reports also scored significantly lower than editorials on the same topics from the same news outlets. GIS proved better at discriminating among texts than did alternative formulae. In a behavioral experiment with closely matched text pairs, people randomly assigned to high-GIS versions scored significantly higher on knowledge and comprehension.